CLAIMS: 

We claim: 

1 . A method of evaluating the quality of voice input recognition by a voice portal, 
said method comprising the steps of: 

extracting a current grammar from the voice portal; 

generating a test input for the current grammar, the test input including a test 
pattern and a set of active grammars for the current grammar; 
providing the test input to the voice portal; 

analyzing the test pattern with respect to the set of active grammars with a 
speech recognition engine in the voice portal; and 

deriving a measure of quality of recognition for the current grammar. 

2. The method of claim 1 , wherein said deriving step includes the step of deriving a 
confidence level and a set of n-best results for the test input, and further comprising the 
steps of: 

comparing the confidence level and set of n-best results for the test input with an 
expected value to assess the measure of quality of recognition. 

3. The method of claim 1 , further comprising the steps of: 

modifying the current grammar to create a modified grammar if the measure of 
quality of recognition for the current grammar deviates from a pre-determined range. 
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4. The method of claim 3, further comprising the steps of: 

(i) generating a test input for the modified grammar, the test input including a 
test pattern and a set of active grammars for the modified grammar; 

(ii) providing the test input for the modified grammar to the voice portal; 

(iii) analyzing the test pattern for the modified grammar with respect to the set 
of active grammars corresponding to the modified grammar with the speech recognition 
engine in the voice portal; 

(iv) deriving a measure of quality of recognition of the modified grammar; and 

(v) re-modifying the modified grammar and repeating steps (i) through (iv) 
until the measure of quality of recognition of the modified grammar does not deviate 
from a pre-determined range. 

5. The method of claim 1 , further comprising the steps of: 

modifying the test pattern to emulate one or more user voices prior to entering 
the test input into the voice portal. 

6. The method of claim 1 , further comprising the steps of: 
modifying the test pattern to emulate the influence of one or more 

communications network qualities prior to entering the test input into the voice portal. 
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7. A machine readable storage having stored thereon a computer program for 
evaluating the quality of voice input recognition by a voice portal, said computer 
program comprising a routine set of instructions which when executed by a machine 
cause the machine to perform the steps of: 

extracting a current grammar from the voice portal; 

generating a test input for the current grammar, the test input including a test 

pattern and a set of active grammars for the current grammar; 
providing the test input to the voice portal- 
analyzing the test pattern with respect to the set of active grammars with a 

speech recognition engine in the voice portal; and 

deriving a measure of quality of recognition for the current grammar. 

8. The machine readable storage of claim 7, wherein said deriving step includes the 
step of deriving a confidence level and a set of n-best results for the test input, and 
further causing said machine to perform the steps of: 

comparing the confidence level and set of n-best results for the test input with an 
expected value to assess the measure of quality of recognition. 

9. The machine readable storage of claim 7, further causing said machine to 
perform the steps of: 

modifying the current grammar to create a modified grammar if the measure of 
quality of recognition for the current grammar deviates from a pre-determined range. 
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10. The machine readable storage of claim 9, further causing said machine to 
perform the steps of: 

(i) generating a test input for the modified grammar, the test input including a 
test pattern and a set of active grammars for the modified grammar; 

(ii) providing the test input for the modified grammar to the voice portal; 

(iii) analyzing the test pattern for the modified grammar with respect to the set 
of active grammars corresponding to the modified grammar with the speech recognition 
engine in the voice portal; 

(iv) deriving a measure of quality of recognition of the modified grammar; and 

(v) re-modifying the modified grammar and repeating steps (i) through (iv) 
until the measure of quality of recognition of the modified grammar does not deviate 
from a pre-determined range. 

1 1 . The machine readable storage of claim 7, further causing said machine to 
perform the steps of: 

modifying the test pattern to emulate one or more user voices prior to entering 
the test input into the voice portal. 

12. The machine readable storage of claim 7, causing said machine to perform the 
steps of: 

modifying the test pattern to emulate the influence of one or more 
communications network qualities prior to entering the test input into the voice portal. 
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1 3. A system for evaluating the quality of voice input recognition by a voice portal 
having a speech recognition engine, comprising: 

an analysis interface for extracting a set of current grammars from the voice 

portal; 

a test pattern generator for generating a test input for each current grammar, the 
test input including a test pattern and a set of active grammars corresponding to each 
current grammar; 

a text-to-speech engine for entering each test pattern into the voice portal; 

a results collector for analyzing each test pattern entered into the voice portal 
with the speech recognition engine against the set of active grammars corresponding to 
the current grammar for said test pattern; and 

a results analyzer for deriving a set of statistics of a quality of recognition of each 
current grammar. 

1 4. The system of claim 1 3, 

wherein the set of statistics includes a confidence level and a set of n-best 
results for each test input, and 

wherein the results analyzer is configured to compare the confidence level and 
set of n-best results for each test input with an expected value to assess the quality of 
recognition of each current grammar having said test input with respect to its 
corresponding set of active grammars. 
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1 5. The system of claim 1 3, 

wherein the test pattern generator is configured to modify each test pattern to 
emulate one or more user voices prior to entering the test input into the voice portal. 

16. The method of claim 1 , further comprising the steps of: 

wherein the test pattern generator is configured to modify each test pattern to 
emulate the influence of one or more communications network qualities prior to entering 
the test input into the voice portal. 
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